Info-concierge: Proactive Multi-modal Interaction through Mind Probing

To close the wide gap between the exploding information world and individual human knowledge, we have investigated a multi-modal interaction strategy called Mind Probing in which a system proactively acts on the user and then estimates his/her internal state by analyzing his/her reaction. We have also developed a system, Info-concierge, that can probe the latent interest of the user, make him/her aware of it, and proactively provide sensible information. The system uses the core techniques of Mind Probing through speech and gaze recognition. In this paper, we present the interaction flow, the core techniques, and a field trial of Info-concierge.